User-Centered Design and Implementation of an Interoperable FHIR Application for Pediatric Pneumonia Prognostication in a Randomized Trial.

OBJECTIVE: To support a pragmatic, electronic health record (EHR)-based randomized controlled trial, we applied user-centered design (UCD) principles, evidence-based risk communication strategies, and interoperable software architecture to design, test, and deploy a prognostic tool for children in emergency departments (EDs) with pneumonia. METHODS: Risk for severe in-hospital outcomes was estimated using a validated ordinal logistic regression model to classify pneumonia severity. To render the results usable for ED clinicians, we created an integrated SMART on FHIR web application built for interoperable use in two pediatric EDs using different EHR vendors: Epic and Cerner. We followed a UCD framework, including problem analysis and user research, conceptual design and early prototyping, user interface development, formative evaluation, and post-deployment summative evaluation. RESULTS: Problem analysis and user research from 39 clinicians and nurses revealed user preferences for risk aversion, accessibility, and timing of risk communication. Early prototyping and iterative design incorporated evidence-based design principles, including numeracy, risk framing, and best-practice visualization techniques. After rigorous unit and end-to-end testing, the application was successfully deployed in both EDs, which facilitatd enrollment, randomization, model visualization, data capture, and reporting for trial purposes. CONCLUSIONS: The successful implementation of a custom application for pneumonia prognosis and clinical trial support in two health systems on different EHRs demonstrates the importance of UCD, adherence to modern clinical data standards, and rigorous testing. Key lessons included the need for understanding users' real-world needs, regular knowledge management, application maintenance, and the recognition that FHIR applications require careful configuration for interoperability.

Adapting Cognitive Task Analysis Methods for Use in a Large Sample Simulation Study of High-Risk Healthcare Events.

Cognitive task analysis (CTA) methods are traditionally used to conduct small-sample, in-depth studies. In this case study, CTA methods were adapted for a large multi-site study in which 102 anesthesiologists worked through four different high-fidelity simulated high-consequence incidents. Cognitive interviews were used to elicit decision processes following each simulated incident. In this paper, we highlight three practical challenges that arose: (1) standardizing the interview techniques for use across a large, distributed team of diverse backgrounds; (2) developing effective training; and (3) developing a strategy to analyze the resulting large amount of qualitative data. We reflect on how we addressed these challenges by increasing standardization, developing focused training, overcoming social norms that hindered interview effectiveness, and conducting a staged analysis. We share findings from a preliminary analysis that provides early validation of the strategy employed. Analysis of a subset of 64 interview transcripts using a decompositional analysis approach suggests that interviewers successfully elicited descriptions of decision processes that varied due to the different challenges presented by the four simulated incidents. A holistic analysis of the same 64 transcripts revealed individual differences in how anesthesiologists interpreted and managed the same case.

Decision-making in anesthesiology: will artificial intelligence make intraoperative care safer?

PURPOSE OF REVIEW: This article explores the impact of recent applications of artificial intelligence on clinical anesthesiologists' decision-making. RECENT FINDINGS: Naturalistic decision-making, a rich research field that aims to understand how cognitive work is accomplished in complex environments, provides insight into anesthesiologists' decision processes. Due to the complexity of clinical work and limits of human decision-making (e.g. fatigue, distraction, and cognitive biases), attention on the role of artificial intelligence to support anesthesiologists' decision-making has grown. Artificial intelligence, a computer's ability to perform human-like cognitive functions, is increasingly used in anesthesiology. Examples include aiding in the prediction of intraoperative hypotension and postoperative complications, as well as enhancing structure localization for regional and neuraxial anesthesia through artificial intelligence integration with ultrasound. SUMMARY: To fully realize the benefits of artificial intelligence in anesthesiology, several important considerations must be addressed, including its usability and workflow integration, appropriate level of trust placed on artificial intelligence, its impact on decision-making, the potential de-skilling of practitioners, and issues of accountability. Further research is needed to enhance anesthesiologists' clinical decision-making in collaboration with artificial intelligence.

Decision-Making During High-Risk Events: A Systematic Literature Review.

Effective decision-making in crisis events is challenging due to time pressure, uncertainty, and dynamic decisional environments. We conducted a systematic literature review in PubMed and PsycINFO, identifying 32 empiric research papers that examine how trained professionals make naturalistic decisions under pressure. We used structured qualitative analysis methods to extract key themes. The studies explored different aspects of decision-making across multiple domains. The majority (19) focused on healthcare; military, fire and rescue, oil installation, and aviation domains were also represented. We found appreciable variability in research focus, methodology, and decision-making descriptions. We identified five main themes: (1) decision-making strategy, (2) time pressure, (3) stress, (4) uncertainty, and (5) errors. Recognition-primed decision-making (RPD) strategies were reported in all studies that analyzed this aspect. Analytical strategies were also prominent, appearing more frequently in contexts with less time pressure and explicit training to generate multiple explanations. Practitioner experience, time pressure, stress, and uncertainty were major influencing factors. Professionals must adapt to the time available, types of uncertainty, and individual skills when making decisions in high-risk situations. Improved understanding of these decisional factors can inform evidence-based enhancements to training, technology, and process design.

Medication Safety Amid Technological Change: Usability Evaluation to Inform Inpatient Nurses' Electronic Health Record System Transition.

BACKGROUND: Electronic health record (EHR) system transitions are challenging for healthcare organizations. High-volume, safety-critical tasks like barcode medication administration (BCMA) should be evaluated, yet standards for ensuring safety during transition have not been established. OBJECTIVE: Identify risks in common and problem-prone medication tasks to inform safe transition between BCMA systems and establish benchmarks for future system changes. DESIGN: Staff nurses completed simulation-based usability testing in the legacy system (R1) and new system pre- (R2) and post-go-live (R3). Tasks included (1) Hold/Administer, (2) IV Fluids, (3) PRN Pain, (4) Insulin, (5) Downtime/PRN, and (6) Messaging. Audiovisual recordings of task performance were systematically analyzed for time, navigation, and errors. The System Usability Scale measured perceived usability and satisfaction. Post-simulation interviews captured nurses' qualitative comments and perceptions of the systems. PARTICIPANTS: Fifteen staff nurses completed 2-3-h simulation sessions. Eleven completed both R1 and R2, and seven completed all three rounds. Clinical experience ranged from novice (< 1 year) to experienced (> 10 years). Practice settings included adult and pediatric patient populations in ICU, stepdown, and acute care departments. MAIN MEASURES: Task completion rates/times, safety and non-safety-related use errors (interaction difficulties), and user satisfaction. KEY RESULTS: Overall success rates remained relatively stable in all tasks except two: IV Fluids task success increased substantially (R1: 17%, R2: 54%, R3: 100%) and Downtime/PRN task success decreased (R1: 92%, R2: 64%, R3: 22%). Among the seven nurses who completed all rounds, overall safety-related errors decreased 53% from R1 to R3 and 50% from R2 to R3, and average task times for successfully completed tasks decreased 22% from R1 to R3 and 38% from R2 to R3. CONCLUSIONS: Usability testing is a reasonable approach to compare different BCMA tasks to anticipate transition problems and establish benchmarks with which to monitor and evaluate system changes going forward.

Evaluation of inpatient medication guidance from an artificial intelligence chatbot.

PURPOSE: To analyze the clinical completeness, correctness, usefulness, and safety of chatbot and medication database responses to everyday inpatient medication-use questions. METHODS: We evaluated the responses from an artificial intelligence chatbot, a medication database, and clinical pharmacists to 200 real-world medication-use questions. Answer quality was rated by a blinded group of pharmacists, providers, and nurses. Chatbot and medication database responses were deemed "acceptable" if the mean reviewer rating was within 3 points of the mean rating for pharmacists' answers. We used descriptive statistics for reviewer ratings and Kendall's coefficient to evaluate interrater agreement. RESULTS: The medication database generated responses to 194 (97%) questions, with 88% considered acceptable for clinical correctness, 76% considered acceptable for completeness, 83% considered acceptable for safety, and 81% considered acceptable for usefulness compared to pharmacists' answers. The chatbot responded to only 160 (80%) questions, with 85% considered acceptable for clinical correctness, 65% considered acceptable for completeness, 71% considered acceptable for safety, and 68% considered acceptable for usefulness. CONCLUSION: Traditional search methods using a drug database provide more clinically correct, complete, safe, and useful answers than a chatbot. When the chatbot generated a response, the clinical correctness was similar to that of a drug database; however, it was not rated as favorably for clinical completeness, safety, or usefulness. Our results highlight the need for ongoing training and continued improvements to artificial intelligence chatbots for them to be incorporated reliably into the clinical workflow. With continued improvement in chatbot functionality, chatbots could be a useful pharmacist adjunct, providing healthcare providers with quick and reliable answers to medication-use questions.

The decision aid is the easy part: workflow challenges of shared decision making in cancer care.

Delivering high-quality, patient-centered cancer care remains a challenge. Both the National Academy of Medicine and the American Society of Clinical Oncology recommend shared decision making to improve patient-centered care, but widespread adoption of shared decision making into clinical care has been limited. Shared decision making is a process in which a patient and the patient's health-care professional weigh the risks and benefits of different options and come to a joint decision on the best course of action for that patient on the basis of their values, preferences, and goals for care. Patients who engage in shared decision making report higher quality of care, whereas patients who are less involved in these decisions have statistically significantly higher decisional regret and are less satisfied. Decision aids can improve shared decision making-for example, by eliciting patient values and preferences that can then be shared with clinicians and by providing patients with information that may influence their decisions. However, integrating decision aids into the workflows of routine care is challenging. In this commentary, we explore 3 workflow-related barriers to shared decision making: the who, when, and how of decision aid implementation in clinical practice. We introduce readers to human factors engineering and demonstrate its potential value to decision aid design through a case study of breast cancer surgical treatment decision making. By better employing the methods and principles of human factors engineering, we can improve decision aid integration, shared decision making, and ultimately patient-centered cancer outcomes.

Antibiotic clinical decision support for pneumonia in the ED: A randomized trial.

BACKGROUND: Electronic health record-based clinical decision support (CDS) is a promising antibiotic stewardship strategy. Few studies have evaluated the effectiveness of antibiotic CDS in the pediatric emergency department (ED). OBJECTIVE: To compare the effectiveness of antibiotic CDS vs. usual care for promoting guideline-concordant antibiotic prescribing for pneumonia in the pediatric ED. DESIGN: Pragmatic randomized clinical trial. SETTING AND PARTICIPANTS: Encounters for children (6 months-18 years) with pneumonia presenting to two tertiary care children s hospital EDs in the United States. INTERVENTION: CDS or usual care was randomly assigned during 4-week periods within each site. The CDS intervention provided antibiotic recommendations tailored to each encounter and in accordance with national guidelines. MAIN OUTCOME AND MEASURES: The primary outcome was exclusive guideline-concordant antibiotic prescribing within the first 24 h of care. Safety outcomes included time to first antibiotic order, encounter length of stay, delayed intensive care, and 3- and 7-day revisits. RESULTS: 1027 encounters were included, encompassing 478 randomized to usual care and 549 to CDS. Exclusive guideline-concordant prescribing did not differ at 24 h (CDS, 51.7% vs. usual care, 53.3%; odds ratio [OR] 0.94 [95% confidence interval [CI]: 0.73, 1.20]). In pre-specified stratified analyses, CDS was associated with guideline-concordant prescribing among encounters discharged from the ED (74.9% vs. 66.0%; OR 1.53 [95% CI: 1.01, 2.33]), but not among hospitalized encounters. Mean time to first antibiotic was shorter in the CDS group (3.0 vs 3.4 h; p = .024). There were no differences in safety outcomes. CONCLUSIONS: Effectiveness of ED-based antibiotic CDS was greatest among those discharged from the ED. Longitudinal interventions designed to target both ED and inpatient clinicians and to address common implementation challenges may enhance the effectiveness of CDS as a stewardship tool.

Understanding Patient and Clinician Reported Nonroutine Events in Ambulatory Surgery.

OBJECTIVE: Nonroutine events (NREs, i.e., deviations from optimal care) can identify care process deficiencies and safety risks. Nonroutine events reported by clinicians have been shown to identify systems failures, but this methodology fails to capture the patient perspective. The objective of this prospective observational study is to understand the incidence and nature of patient- and clinician-reported NREs in ambulatory surgery. METHODS: We interviewed patients about NREs that occurred during their perioperative care using a structured interview tool before discharge and in a 7-day follow-up call. Concurrently, we interviewed the clinicians caring for these patients immediately postoperatively to collect NREs. We trained 2 experienced clinicians and 2 patients to assess and code each reported NRE for type, theme, severity, and likelihood of reoccurrence (i.e., likelihood that the same event would occur for another patient). RESULTS: One hundred one of 145 ambulatory surgery cases (70%) contained at least one NRE. Overall, 214 NREs were reported-88 by patients and 126 by clinicians. Cases containing clinician-reported NREs were associated with increased patient body mass index ( P = 0.023) and lower postcase patient ratings of being treated with respect ( P = 0.032). Cases containing patient-reported NREs were associated with longer case duration ( P = 0.040), higher postcase clinician frustration ratings ( P < 0.001), higher ratings of patient stress ( P = 0.019), and lower patient ratings of their quality of life ( P = 0.010), of the quality of clinician teamwork ( P = 0.010), being treated with respect ( P = 0.003), and being listened to carefully ( P = 0.012). Trained patient raters evaluated NRE severity significantly higher than did clinician raters ( P < 0.001), while clinicians rated recurrence likelihood significantly higher than patients for both clinician ( P = 0.032) and patient-reported NREs ( P = 0.001). CONCLUSIONS: Both patients and clinicians readily report events during clinical care that they believe deviate from optimal care expectations. These 2 primary stakeholders in safe, high-quality surgical care have different experiences and perspectives regarding NREs. The combination of patient- and clinician-reported NREs seems to be a promising patient-centered method of identifying healthcare system deficiencies and opportunities for improvement.

A pilot study to determine the incidence, type, and severity of non-routine events in neonates undergoing gastrostomy tube placement.

BACKGROUND: Non-routine events (NRE) are defined as any suboptimal occurrences in a process being measured in the opinion of the reporter and comes from the field of human factors engineering. These typically occur well up-stream of an adverse event and NRE measurement has not been applied to the complex context of neonatal surgery. We sought to apply this novel safety event measurement methodology to neonates in the NICU undergoing gastrostomy tube placement. METHODS: A prospective pilot study was conducted between November 2016 and August 2020 in the Level IV NICU and the pediatric operating rooms of an urban academic children's hospital to determine the incidence, severity, impact, and contributory factors of clinician-reported non-routine events (NREs, i.e., deviations from optimal care) and 30-day NSQIP occurrences in neonates receiving a G-tube. RESULTS: Clinicians reported at least one NRE in 32 of 36 (89%) G-tube cases, averaging 3.0 (Standard deviation: 2.5) NRE reports per case. NSQIP-P review identified 7 cases (19%) with NSQIP-P occurrences and each of these cases had multiple reported NREs. One case in which NREs were not reported was without NSQIP-P occurrences. The odds ratio of having a NSQIP-P occurrence with the presence of an NRE was 0.695 (95% CI 0.06-17.04). CONCLUSION: Despite being considered a "simple" operation, >80% of neonatal G-tube placement operations had at least one reported NRE by an operative team member. In this pilot study, NRE occurrence was not significantly associated with the subsequent reporting of an NSQIP-P occurrence. Understanding contributory factors of NREs that occur in neonatal surgery may promote surgical safety efforts and should be evaluated in larger and more diverse populations. LEVEL OF EVIDENCE: IV.

Reliability of simulation-based assessment for practicing physicians: performance is context-specific.

INTRODUCTION: Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established. METHODS: Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters. RESULTS: The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability. CONCLUSIONS: The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.

Prevalence of burnout and its relationship to health status and social support in more than 1000 subspecialty anesthesiologists.

BACKGROUND: Physician burnout may be at 'epidemic' proportions due to factors associated with modern healthcare practice and technology. Practice attributes vary appreciably among subspecialists. Understanding burnout incidence and its associated factors could illuminate potential causes and interventions. We evaluated burnFout, mental and physical health, and social support and coping skills in acute and chronic pain physicians and pediatric and cardiac anesthesiologists. METHODS: We administered the Maslach Burnout Inventory Human Services Survey (MBI-HSS), a two-item self-identified burnout measure, the Veterans RAND 12-item Health Survey and the Social Support and Personal Coping Survey to subspecialty society members practicing acute and chronic pain management, pediatric anesthesiology and cardiac anesthesiology. Multivariable regression analysis compared the groups, and adjusted burnout prevalence was compared with an all-physician and an employed general population sample. RESULTS: Among 1303 participants (response rates 21.6%-35.6% among the subspecialty groups), 43.4% met established burnout criteria (range 30.0%-62.3%). Chronic pain physicians had significantly worse scores (unadjusted) than the other three groups of subspecialty anesthesiologists, the all-physician comparator group and the general population comparator group. Mental health inversely correlated with emotional exhaustion and depersonalization in all groups. Self-identified burnout correlated with the full MBI-HSS (R=0.54; positive predictive value of 0.939 (0.917, 0.955)). Physicians' scores for personal accomplishment were higher than population norms. CONCLUSIONS: This study provides data on burnout prevalence and associated demographic, health and social factors in subspecialist anesthesiologists. Chronic pain anesthesiologists had significantly greater burnout than the other groups. The self-identified burnout metric performed well and may be an attractive alternative to the full MBI-HSS.

Defining the Epidemiology of Safety Risks in Neonatal Intensive Care Unit Patients Requiring Surgery.

OBJECTIVE: The aim of the study was to determine the incidence, type, severity, preventability, and contributing factors of nonroutine events (NREs)-events perceived by care providers or skilled observers as a deviations from optimal care based on the clinical situation-in the perioperative (i.e., preoperative, operative, and postoperative) care of surgical neonates in the neonatal intensive care unit and operating room. METHODS: A prospective observational study of noncardiac surgical neonates, who received preoperative and postoperative neonatal intensive care unit care, was conducted at an urban academic children's hospital between November 1, 2016, and March 31, 2018. One hundred twenty-nine surgical cases in 109 neonates were observed. The incidence and description of NREs were collected via structured researcher-administered survey tool of involved clinicians. Primary measurements included clinicians' ratings of NRE severity and contributory factors and trained research assistants' ratings of preventability. RESULTS: One or more NREs were reported in 101 (78%) of 129 observed cases for 247 total NREs. Clinicians reported 2 (2) (median, interquartile range) NREs per NRE case with a maximum severity of 3 (1) (possible range = 1-5). Trained research assistants rated 47% of NREs as preventable and 11% as severe and preventable. The relative risks for National Surgical Quality Improvement Program - pediatric major morbidity and 30-day mortality were 1.17 (95% confidence interval = 0.92-1.48) and 1.04 (95% confidence interval = 1.00-1.08) in NRE cases versus non-NRE cases. CONCLUSIONS: The incidence of NREs in neonatal perioperative care at an academic children's hospital was high and of variable severity with a myriad of contributory factors.

Incidence and Classification of Nonroutine Events during Anesthesia Care.

BACKGROUND: A nonroutine event is any aspect of clinical care perceived by clinicians or trained observers as a deviation from optimal care based on the context of the clinical situation. The authors sought to delineate the incidence and nature of intraoperative nonroutine events during anesthesia care. METHODS: The authors prospectively collected audio, video, and relevant clinical information on 556 cases at three academic hospitals from 1998 to 2004. In addition to direct observation, anesthesia providers were surveyed for nonroutine event occurrence and details at the end of each study case. For the 511 cases with reviewable video, 400 cases had no reported nonroutine events and 111 cases had at least one nonroutine event reported. Each nonroutine event was analyzed by trained anesthesiologists. Rater reliability assessment, comparisons (nonroutine event vs. no event) of patient and case variables were performed. RESULTS: Of 511 cases, 111 (21.7%) contained 173 nonroutine events; 35.1% of event-containing cases had more than one nonroutine event. Of the 173 events, 69.4% were rated as having patient impact and 12.7% involved patient injury. Longer case duration (25th vs. 75th percentile; odds ratio, 1.83; 95% CI, 1.15 to 2.93; P = 0.032) and presence of a comorbid diagnosis (odds ratio, 2.14; 95% CI, 1.35 to 3.40; P = 0.001) were associated with nonroutine events. Common contributory factors were related to the patient (63.6% [110 of 173]) and anesthesia provider (59.0% [102 of 173]) categories. The most common patient impact events involved the cardiovascular system (37.4% [64 of 171]), airway (33.3% [57 of 171]), and human factors, drugs, or equipment (31.0% [53 of 171]). CONCLUSIONS: This study describes characteristics of intraoperative nonroutine events in a cohort of cases at three academic hospitals. Nonroutine event-containing cases were commonly associated with patient impact and injury. Thus, nonroutine event monitoring in conjunction with traditional error reporting may enhance our understanding of potential intraoperative failure modes to guide prospective safety interventions.

CYP2D6 Genotype-guided Metoprolol Therapy in Cardiac Surgery Patients: Rationale and Design of the Pharmacogenetic-guided Metoprolol Management for Postoperative Atrial Fibrillation in Cardiac Surgery (PREEMPTIVE) Pilot Study.

OBJECTIVES: The Preemptive Pharmacogenetic-guided Metoprolol Management for Atrial Fibrillation in Cardiac Surgery (PREEMPTIVE) pilot trial aims to use existing institutional resources to develop a process for integrating CYP2D6 pharmacogenetic test results into the patient electronic health record, to develop an evidence-based clinical decision support tool to facilitate CYP2D6 genotype-guided metoprolol administration in the cardiac surgery setting, and to determine the impact of implementing this CYP2D6 genotype-guided integrated approach on the incidence of postoperative atrial fibrillation (AF), provider, and cost outcomes. DESIGN: One-arm Bayesian adaptive design clinical trial. SETTING: Single center, university hospital. PARTICIPANTS: The authors will screen (including CYP2D6 genotype) up to 600 (264 ± 144 expected under the adaptive design) cardiac surgery patients, and enroll up to 200 (88 ± 48 expected) poor, intermediate, and ultrarapid CYP2D6 metabolizers over a period of 2 years at a tertiary academic center. INTERVENTIONS: All consented and enrolled patients will receive the intervention of CYP2D6 genotype-guided metoprolol management based on CYP2D6 phenotype classified as a poor, intermediate, extensive (normal), or ultrarapid metabolizer. MEASUREMENTS AND MAIN RESULTS: The primary outcome will be the incidence of postoperative AF. Secondary outcomes relating to rates of CYP2D6 genotype-guided prescription changes, costs, lengths of stay, and implementation metrics also will be investigated. CONCLUSIONS: The PREEMPTIVE pilot study is the first perioperative pilot trial to provide essential information for the design of a future, large-scale trial comparing CYP2D6 genotype-guided metoprolol management with a nontailored strategy in terms of managing AF. In addition, secondary outcomes regarding implementation, clinical benefit, safety, and cost-effectiveness in patients undergoing cardiac surgery will be examined.